Seamless switching of scalable video bitstreams

ABSTRACT

In accordance with certain aspects of the present invention, methods and apparatuses are provided according to a seamless bitstream switching scheme. The seamless switching scheme can be used with scalable video bitstreams, and can take advantage of both the high coding efficiency of non-scalable bitstreams and the flexibility of scalable bitstreams. Small bandwidth fluctuations can be accommodated by the scalability of the bitstreams, while large bandwidth fluctuations can be tolerated by switching among scalable bitstreams. The flexible and effective scheme for seamless switching among scalable bitstreams significantly improves the efficiency of scalable video coding over a broad bit rate range.

RELATED PATENT APPLICATIONS

[0001] This U.S. Non-provisional Application for Letters Patent furtherclaims the benefit of priority from, and hereby incorporates byreference the entire disclosure of, co-pending U.S. ProvisionalApplication for Letters Patent Serial No. 60/351,891, filed Jan. 25,2002.

[0002] Furthermore, this U.S. Non-provisional Application for LettersPatent is related to a co-pending application Ser. No. ______(Attorney's Docket Number MS1-1220US), filed Jun. 27, 2002, and titled“Methods And Apparatuses For Use In Switching Between Streaming VideoBitstreams”.

TECHNICAL FIELD

[0003] This invention relates to data bitstreams, and more particularlyto methods and apparatuses for switching between different bitstreams.

BACKGROUND

[0004] With steady growth of access bandwidth, more and more Internetapplications start to use streaming audio and video contents. Since thecurrent Internet is inherently a heterogeneous and dynamical best-effortnetwork, channel bandwidth usually fluctuates in a wide range from bitrate below 64 kbps to well above 1 Mbps. This brings great challenges tovideo coding and streaming technologies in providing a smooth playbackexperience and best available video quality to the users. To deal withthe network bandwidth variations, two main approaches, namely, switchingamong multiple non-scalable bitstreams and streaming with a singlescalable bitstream, have been extensively investigated in recent years.

[0005] In the first approach, a video sequence is compressed intoseveral non-scalable bitstreams at different bit rates. Some specialframes, known as key frames, are either compressed without prediction orcoded with an extra switching bitstream. Key frames provide accesspoints to switch among these bitstreams to fit in the availablebandwidth. One advantage of this method is the high coding efficiencywith non-scalable bitstreams. However, due to limitation in both thenumber of bitstreams and switching points, this method only providescoarse and sluggish capability in adapting to channel bandwidthvariations.

[0006] In the second approach, a video sequence is compressed into asingle scalable bitstream, which can be truncated flexibly to adapt tobandwidth variations. Among numerous scalable coding techniques, MPEG-4Fine Granularity Scalable (FGS) coding has become prominent due to itsfine-grain scalability. Since the enhancement bitstream can be truncatedarbitrarily in any frame, FGS provides a remarkable capability inreadily and precisely adapting to channel bandwidth variations. However,low coding efficiency is the vital disadvantage that prevents FGS frombeing widely deployed in video streaming applications. Progressive FineGranularity Scalable (PFGS) coding scheme is a significant improvementover FGS by introducing two prediction loops with different qualityreferences. On the other hand, since only one high quality reference isused in enhancement layer coding, most coding efficiency gain appearswithin a certain bit rate range around the high quality reference.Generally, with today's technologies, there is still a coding efficiencyloss compared with the non-scalable case at fixed bit rates.

[0007] Hence, there is a need for improved method and apparatuses foruse in switching bitstreams.

SUMMARY

[0008] A method is provided for switching between a first bitstream anda second bitstream, wherein the second bitstream provides a higher datarate than the first bitstream. The method includes identifying aswitching up period associated with the first and second bitstreams. Themethod also includes establishing a third bitstream during the switchingup period, the third bitstream being configured to carry additional dataassociated with a switch up from the first bitstream to the secondbitstream. The method further includes switching up from the firstbitstream to the second bitstream during the switching up period. Themethod may also support switching down.

[0009] An apparatus that is suitable for encoding a first bitstream anda second bitstream, wherein the second bitstream provides a higher datarate than the first bitstream is also provided. Here, the apparatusincludes an interface that is configurable to selectively output thefirst and second bitstreams. The apparatus also includes logic that isconfigured to selectively encode a plurality of frames of a videosequence within the first bitstream as first encoded data that is atleast partially transmittable by the interface in a base layer portionof the first bitstream and the plurality of frames of the video sequencewithin the second bitstream as second encoded data that is at leastpartially transmittable by the interface in a base layer portion of thesecond bitstream. The logic also establishes a plurality of temporallyspaced switching up periods associated with the first and secondbitstreams, during which, the logic causes the interface to output athird bitstream that is configured to carry additional data associatedwith a switch up from the first bitstream to the second bitstream. Theapparatus may also support switching down.

BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The present invention is illustrated by way of example and notlimitation in the figures of the accompanying drawings. The same numbersare used throughout the MPFGS figures to reference like componentsand/or features.

[0011]FIG. 1 is a block diagram depicting an exemplary computingenvironment that is suitable for use with certain implementations of thepresent invention.

[0012]FIG. 2 is a diagram illustratively depicting switching up andswitching down between scalable bitstreams, in accordance with certainexemplary implementations of the present invention.

[0013] A FIG. 3 is a block diagram depicting an encoding framework, inaccordance with certain exemplary implementations of the presentinvention.

[0014]FIG. 4 is a block diagram depicting base layer decoder, inaccordance with certain exemplary implementations of the presentinvention

[0015]FIG. 5 is a diagram illustratively depicting switching framecoding techniques, in accordance with certain exemplary implementationsof the present invention.

[0016] FIGS. 6A-B, 7A-B, and 8A-B are graph depicting some experimentalresults for methods and apparatuses, in accordance with certainexemplary implementations of the present invention.

[0017]FIG. 9 is a block diagram depicting a conventional SP codingscheme.

[0018]FIG. 10 is a block diagram depicting an enhanced SP coding scheme,in accordance with certain exemplary implementations of the presentinvention.

[0019]FIG. 11 is a graph depicting some experimental results for methodsand apparatuses, in accordance with certain exemplary implementations ofthe present invention.

DETAILED DESCRIPTION

[0020] In accordance with certain aspects of the present invention,methods and apparatuses are provided according to a seamless switchingscheme. The seamless switching scheme is for scalable video bitstreams,and takes advantage of both the high coding efficiency of non-scalablebitstreams and the flexibility of scalable bitstreams. Small bandwidthfluctuations are accommodated by the scalability of the bitstreams,while large bandwidth fluctuations are tolerated by switching amongscalable bitstreams. The flexible and effective scheme for seamlessswitching among scalable bitstreams significantly improves theefficiency of scalable video coding over a broad bit rate range.

[0021] By way of example, when the channel bandwidth drops below theeffective range of a scalable bitstream operated at higher rates,certain exemplary methods and apparatus are configured to switch at anyframe from a current scalable bitstream to one operated at lower rateswithout sending any overhead bits. Additional bits may be used whenswitching from a scalable bitstream operated at lower rate to oneoperated at higher rate. However, an efficient technique is describedherein to greatly reduce the amount of overhead bits used.

[0022] Experimental results have shown that the exemplary methods andapparatuses significantly outperform both of the convention approachesdescribed in the Background section, e.g., a single scalable bitstreamapproach and an approach of switching among multiple non-scalablebitstreams.

[0023] The next sections introduce some exemplary operating devices,followed by concepts of the seamless switching scheme. An exemplaryseamless switching encoder, for example, configured with two MPFGSbitstreams is then described. Some exemplary methods for switching downand switching up are also described below, followed by some experimentalresults. Additionally, an improved SP coding technique is described

[0024] Exemplary Operational Environments:

[0025] Turning to the drawings, wherein like reference numerals refer tolike elements, the invention is illustrated as being implemented in asuitable computing environment. Although not required, the inventionwill be described in the general context of computer-executableinstructions, such as program modules, being executed by a personalcomputer.

[0026] Generally, program modules include routines, programs, objects,components, data structures, etc. that perform particular tasks orimplement particular abstract data types. Those skilled in the art willappreciate that the invention may be practiced with other computersystem configurations, including hand-held devices, multi-processorsystems, microprocessor based or programmable consumer electronics,network PCs, minicomputers, mainframe computers, portable communicationdevices, and the like.

[0027] The invention may also be practiced in distributed computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network. In a distributed computingenvironment, program modules may be located in both local and remotememory storage devices.

[0028]FIG. 1 illustrates an example of a suitable computing environment120 on which the subsequently described systems, apparatuses and methodsmay be implemented. Exemplary computing environment 120 is only oneexample of a suitable computing environment and is not intended tosuggest any limitation as to the scope of use or functionality of theimproved methods and systems described herein. Neither should computingenvironment 120 be interpreted as having any dependency or requirementrelating to any one or combination of components illustrated incomputing environment 120.

[0029] The improved methods and systems herein are operational withnumerous other general purpose or special purpose computing systemenvironments or configurations. Examples of well known computingsystems, environments, and/or configurations that may be suitableinclude, but are not limited to, personal computers, server computers,thin clients, thick clients, hand-held or laptop devices, multiprocessorsystems, microprocessor-based systems, set top boxes, programmableconsumer electronics, network PCs, minicomputers, mainframe computers,distributed computing environments that include any of the above systemsor devices, and the like.

[0030] As shown in FIG. 1, computing environment 120 includes ageneral-purpose computing device in the form of a computer 130. Thecomponents of computer 130 may include one or more processors orprocessing units 132, a system memory 134, and a bus 136 that couplesvarious system components including system memory 134 to processor 132.

[0031] Bus 136 represents one or more of any of several types of busstructures, including a memory bus or memory controller, a peripheralbus, an accelerated graphics port, and a processor or local bus usingany of a variety of bus architectures. By way of example, and notlimitation, such architectures include Industry Standard Architecture(ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA)bus, Video Electronics Standards Association (VESA) local bus, andPeripheral Component Interconnects (PCI) bus also known as Mezzaninebus.

[0032] Computer 130 typically includes a variety of computer readablemedia. Such media may be any available media that is accessible bycomputer 130, and it includes both volatile and non-volatile media,removable and non-removable media.

[0033] In FIG. 1, system memory 134 includes computer readable media inthe form of volatile memory, such as random access memory (RAM) 140,and/or non-volatile memory, such as read only memory (ROM) 138. A basicinput/output system (BIOS) 142, containing the basic routines that helpto transfer information between elements within computer 130, such asduring start-up, is stored in ROM 138. RAM 140 typically contains dataand/or program modules that are immediately accessible to and/orpresently being operated on by processor 132.

[0034] Computer 130 may further include other removable/non-removable,volatile/non-volatile computer storage media. For example, FIG. 1illustrates a hard disk drive 144 for reading from and writing to anon-removable, non-volatile magnetic media (not shown and typicallycalled a “hard drive”), a magnetic disk drive 146 for reading from andwriting to a removable, non-volatile magnetic disk 148 (e.g., a “floppydisk”), and an optical disk drive 150 for reading from or writing to aremovable, non-volatile optical disk 152 such as a CD-ROM/R/RW,DVD-ROM/R/RW/+R/RAM or other optical media. Hard disk drive 144,magnetic disk drive 146 and optical disk drive 150 are each connected tobus 136 by one or more interfaces 154.

[0035] The drives and associated computer-readable media providenonvolatile storage of computer readable instructions, data structures,program modules, and other data for computer 130. Although the exemplaryenvironment described herein employs a hard disk, a removable magneticdisk 148 and a removable optical disk 152, it should be appreciated bythose skilled in the art that other types of computer readable mediawhich can store data that is accessible by a computer, such as magneticcassettes, flash memory cards, digital video disks, random accessmemories (RAMs), read only memories (ROM), and the like, may also beused in the exemplary operating environment.

[0036] A number of program modules may be stored on the hard disk,magnetic disk 148, optical disk 152, ROM 138, or RAM 140, including,e.g., an operating system 158, one or more application programs 160,other program modules 162, and program data 164.

[0037] The improved methods and systems described herein may beimplemented within operating system 158, one or more applicationprograms 160, other program modules 162, and/or program data 164.

[0038] A user may provide commands and information into computer 130through input devices such as keyboard 166 and pointing device 168 (suchas a “mouse”). Other input devices (not shown) may include a microphone,joystick, game pad, satellite dish, serial port, scanner, camera, etc.These and other input devices are connected to the processing unit 132through a user input interface 170 that is coupled to bus 136, but maybe connected by other interface and bus structures, such as a parallelport, game port, or a universal serial bus (USB).

[0039] A monitor 172 or other type of display device is also connectedto bus 136 via an interface, such as a video adapter 174. In addition tomonitor 172, personal computers typically include other peripheraloutput devices (not shown), such as speakers and printers, which may beconnected through output peripheral interface 175.

[0040] Computer 130 may operate in a networked environment using logicalconnections to one or more remote computers, such as a remote computer182. Remote computer 182 may include many or all of the elements andfeatures described herein relative to computer 130.

[0041] Logical connections shown in FIG. 1 are a local area network(LAN) 177 and a general wide area network (WAN) 179. Such networkingenvironments are commonplace in offices, enterprise-wide computernetworks, intranets, and the Internet.

[0042] When used in a LAN networking environment, computer 130 isconnected to LAN 177 via network interface or adapter 186. When used ina WAN networking environment, the computer typically includes a modem178 or other means for establishing communications over WAN 179. Modem178, which may be internal or external, may be connected to system bus136 via the user input interface 170 or other appropriate mechanism.

[0043] Depicted in FIG. 1, is a specific implementation of a WAN via theInternet. Here, computer 130 employs modem 178 to establishcommunications with at least one remote computer 182 via the Internet180.

[0044] In a networked environment, program modules depicted relative tocomputer 130, or portions thereof, may be stored in a remote memorystorage device. Thus, e.g., as depicted in FIG. 1, remote applicationprograms 189 may reside on a memory device of remote computer 182. Itwill be appreciated that the network connections shown and described areexemplary and other means of establishing a communications link betweenthe computers may be used.

[0045] Exemplary Switching Scheme:

[0046] An exemplary seamless switching scheme is provided, whichsignificantly improves the efficiency of scalable video coding over abroad bit rate range by using multiple scalable bitstreams.

[0047] Each scalable bitstream has a base layer with a different bitrate and can best adapt to channel bandwidth variation within a certainbit rate range. If the channel bandwidth is out of this range, thescalable bitstream can be seamlessly switched to another with bettercoding efficiency. Switching from a scalable bitstream operated at lowerbit rates to one operated at higher bit rates is referred to herein asswitching up and the reversion as switching down.

[0048] One problem that is solved is how to flexibly and efficientlyswitch up and down among scalable bitstreams. There are differentrequirements for switching up and switching down, respectively. Whenchannel bandwidth somehow drops, for example, the server has to rapidlyswitch from high bit rate bitstream to low bit rate bitstream to reducepacket loss ratio and maintain smooth video playback.

[0049] In accordance with certain aspects of the present invention, thetechniques for switching down enable the scalable bitstreams to beswitched down at any frame, and reduce/eliminate the use of overheadbits during switching down since the overhead bits tend to increasenetwork traffic and therefore may further deteriorate networkconditions.

[0050] When channel bandwidth increases, a delay is usually needed forthe server to make a reliable decision for switching up. Moreover, insuch as situation, improved/improving network conditions allow foradditional bits to be transmitted. Thus, in accordance with certainexemplary implementations, the switching up scheme attaches an extrabitstream in some frames to support switching up. In order to avoiddrifting errors, the extra bitstream can be formed by losslesslycompressing the mismatches between the reconstructed frames withdifferent bitstreams, which will add significant overhead to theseframes. In accordance with still other implementations, another method,herein referred to as switching frame (SF), is exemplary tosubstantially reduce the overhead bits associated with switching up.

[0051] Reference is now made to FIG. 2, which illustratively depictscertain aspects of an exemplary seamless switching scheme 200 using twoscalable bitstreams, namely bitstream 202 and bitstream 204. Bitstream202 includes a base layer 202 b and an enhanced layer 202 e. Similarly,bitstream 204 includes a base layer 204 b and an enhanced layer 204 e.Base layer 202 b, in this example, has a higher bit rate than base layer204 b.

[0052] An important part of the exemplary seamless switching scheme isto selectively switch up and down between base layers 202 b/204 bwithout causing drifting and/or other like errors. In general, driftingerrors caused, for example, by bitstream truncations, packet losses,transmitted errors, etc., at an enhancement layer, can be recoveredautomatically and gracefully. This is an important feature provided byfine granularity scalable video coding. Therefore, if the exemplaryscheme can successfully switch up and down between the base layers ofmultiple scalable bitstreams, then whole scalable bitstreams alsoachieve such operations.

[0053] When a server is serving client(s) with one scalable bitstream,the base layer of the bitstream is transmitted to the client. However,the enhancement layer bitstream is usually transmitted partiallydependent on the available channel bandwidth. The shaded regions shownin the enhancement layers 202 e and 204 e indicate the part of theirrespective bitstream that has been sent by the server. When channelbandwidth somehow drops below a base layer bit rate of the currentscalable bitstream, then the exemplary switching scheme rapidly switchesdown from high bit rate scalable bitstream 202 to low bit rate scalablebitstream 204 to reduce the packet loss ratio and maintain smooth videoplayback. Therefore, as illustrated by solid arrows with dashed linesbetween the two base layers, the exemplary scheme reserves a path inevery frame for switching down from one base layer (operated at higherbit rate) to another base layer (operated at lower bit rate), as is thecase for base layers 202 b and 204 b. Since there are no overhead bitsneeded for switching down, the server can easily complete the switchingoperation by directly transmitting subsequent frames from the scalablebitstream with lower bit rate base layer, for example, base layer 204 b.

[0054] The exemplary scheme deploys some special points (e.g.,associated with 206 and 208) for switching up, such as illustrated the2^(nd) and (k+1)^(th) frames in FIG. 2. Normally the switching uphappens when the available channel bandwidth increases enough to allowfor use of the higher base layer bit rate. It is reasonable to assumethat the current network condition allows transmitting additionalbitstream for switching up. Therefore, an extra bitstream, asrepresented by the rectangular boxes 206 and 208 with slant stripes inFIG. 2, is utilized at such (special) switching points/periods.Furthermore, in order to effectively reduce the overhead bits forswitching up, the frames at the higher bit rate base layer (e.g., 202 b)are encoded as SF instead of normal P frame at the switching points.

[0055] An Exemplary Switching Scheme Using MPFGS Encoders:

[0056] In this example, either MPEG-4 FGS or PFGS coding is consideredwith regard to the switching scheme. This is not meant as a limitationto be placed on the exemplary methods and apparatuses described herein.Those skilled in the art will recognize that the switching scheme isclearly adaptable to these and other video formats, standards, codingtechniques, as well as various implementations.

[0057] For better coding efficiency and/or other reasons, themacroblock-based PFGS (MPFGS) has been chosen as the basic scalablevideo codec for this example. The MPFGS codec compresses a videosequence into two bitstreams. In each frame, the base layer bitstream isfirst generated by traditional non-scalable coding technique, and thenthe residue between original/predicted DCT coefficients and dequantizedDCT coefficients of the base layer forms the enhancement layer bitstreamwith bit-plane coding techniques. The bit rate of the base layer is thelower bound of the channel bandwidth covered by this scalable bitstream.The enhancement layer bitstream provides fine-grain scalability to adaptto channel bandwidth variations.

[0058] An exemplary arrangement 300 for encoding two scalable bitstreamssuitable for use with the switching scheme is illustrated in FIG. 3.Note that motion estimation modules are omitted for simplicity. Asshown, there are two MPFGS encoders 302 and 304, e.g., as outlined bythe dashed boxes. MPFGS encoders 304 is denoted as LB-MPFGS since itgenerates a scalable bitstream with a lower bit rate base layer 204 b,whereas MPFGS encoders 302 is denoted as HB-MPFGS for higher bit ratebase layer 202 b, accordingly. Portion 308, illustrated between the twoMPFGS encoders, is configured to generate an extra bitstream forswitching up, for example, as described above.

[0059] For convenience in this description, lowercase letters denote animage in pixel domain, and corresponding uppercase letters denote animage in DCT domain. The subscript “b” and “e” indicate the base layerand the enhancement layer, respectively. A tilde “” denotesreconstructed image or DCT coefficients. The symbols “-H” and “-L” areused to distinguish the modules in HB-MPFGS and LB-MPFGS, respectively.

[0060] To ensure that the MPFGS bitstreams are able to be seamlesslyswitched from one to the other, the base layer bitstreams of these twoMPFGS encoders are actually not generated independently. Firstly, motionvectors are estimated in HB-MPFGS and are applied to both HB-MPFGS andLB-MPFGS. Secondly, the video frames to be encoded for the base layer ofLB-MPFGS are the reconstructed base layer frames from HB-MPFGS insteadof the original video frames.

[0061] The original video is first input to HB-MPFGS. Since the motionvectors estimated will be used for both MPFGS encoders, original videoframes are used as reference for estimating integer motion vectors,whereas fractional parts of motion vectors are still estimated byreferencing the reconstructed base layer of HB-MPFGS to maintain thecoding efficiency. There are two references in each MPFGS codec. The lowquality reference {tilde over (r)}_(bH) stored in refBase-H frame bufferis reconstructed from the base layer, whereas the high quality reference{tilde over (r)}_(eH) stored in refEnh-H frame buffer is reconstructedfrom both the base layer and the enhancement layer. The base layer onlyuses low quality reference for prediction and reconstruction, while theenhancement layer can select either the low quality reference or thehigh quality reference, which is decided by mode decision algorithm. Thebase layer bitstream and the enhancement layer bitstream are generatedusing MPEG-4 non-scalable coding and bit plane coding, respectively.

[0062] The LB-MPFGS obtains the motion vectors directly from HB-MPFGSwithout motion estimation. Normally the bit rate of the base layer inHB-MPFGS is much higher than that in LB-MPFGS. To make seamlessswitching from HB-MPFGS bitstream to LB-MPFGS bitstream possible, thereconstructed high quality base layer from HB-MPFGS instead of theoriginal video is input to the LB-MPFGS base layer encoder. In otherwords, the predicted error x_(bL) encoded at LB-MPFGS base layer is thedifference between the reconstructed HB-MPFGS base layer {tilde over(r)}_(bH) and the prediction {tilde over (p)}_(bL). This is similar totranscoding bitstreams from high bit rate to low bit rate. On the otherhand, the predicted error x_(eL) is still calculated from the originalvideo in order to maintain the coding efficiency of the enhancementlayer in LB-MPFGS.

[0063] Switching Down Among Scalable Bitsreams:

[0064] One exemplary technique for switching down is similar totranscoding bitstreams from high bit rate to low bit rate. Normally thebit rate of the base layer in HB-MPFGS is much higher than that inLB-MPFGS. The reconstructed high quality base layer from HB-MPFGSinstead of the original video is input to the LB-MPFGS base layerencoder. Thus, the signal encoded at the LB-MPFGS base layer is thedifference between the reconstructed HB-MPFGS base layer and thetemporal prediction. This is an important feature for achievingswitching down in any frame without any overhead bits.

[0065] When the HB-MPFGS bitstream is being transmitted to the client,the reconstructed reference {tilde over (r)}_(bH), which is the image tobe encoded at the base layer of LB-MPFGS, is very useful for switchingdown. When the network bandwidth somehow drops below the bit rate of theHB-MPFGS base layer, the HB-MPFGS bitstream has to be promptly switchedto the LB-MPFGS bitstream.

[0066] One issue is how to precisely recover the reconstructed image{tilde over (r)}_(bL) to avoid drifting errors. The exemplary schemepresented herein preferably does not transmit an extra bitstream sinceit is likely that the network can hardly tolerate more overhead bits insuch situations. Instead, according to certain features of the scheme, acalculation {tilde over (r)}_(bL) is made directly from {tilde over(r)}_(bH), provided that the low quality prediction {tilde over(p)}_(bL) and quantization parameters of the LB-MPFGS base layer areavailable.

[0067] The quantization parameters of the LB-MPFGS base layer can bereadily encoded into HB-MPFGS bitstream. If the quantization parametersare only adjusted at frame level, only five extra bits, for example, maybe necessary for each frame. Even if the quantization parameters areadjusted at macroblock level, the number of overhead bits is stillrelatively small in HB-MPFGS base layer bitstream.

[0068] Reference is made to FIG. 4, which is a block diagram depictingan exemplary base layer decoder 400, in this case, an HB-MPFGS decoder.The low quality prediction {tilde over (p)}_(bL) is constantly computedby the HB-MPFGS base layer decoder as shown illustrated by the modulesin lox 402. This would increase the complexity of HB-MPFGS decoder, butit would not incur any new overhead bits.

[0069] When scalable bitstream is just switched up to HB-MPFGS, theprediction {tilde over (p)}_(bL) is available in LB-MPFGS. After thenext frame is decoded in HB-MPFGS, the reconstructed reference {tildeover (r)}_(bH) is also available. Since the quantization parameters ofLB-MPFGS are encoded in the HB-MPFGS base layer bitstream, thereconstructed reference {tilde over (r)}_(bL) can be calculated asillustrated in FIG. 4. Furthermore, since the same motion vectors areused at both MPFGS decoders, the HB-MPFGS can readily get the nextprediction {tilde over (p)}_(bL) after motion compensation.

[0070] Hence, the exemplary scheme can switch down in any frame sincethe prediction {tilde over (p)}_(bL) and the quantization parameters ofthe LB-MPFGS base layer are always available in HB-MPFGS. As shown, oneadvantage of this scheme is that no extra overhead bits are needed whenswitching down from HB-MPFGS to LB-MPFGS bitstreams.

[0071] Switching Up Among Scalable Bitstreams:

[0072] The exemplary scheme switches up for one MPFGS bitstream to theother by attaching an extra bitstream in some frames. Since the extraswitching bitstream is usually generated with a lossless method, thiswould cause huge overhead bits in these frames. Therefore, the SFtechnique can be used to greatly reduce the overhead bits for switchingup.

[0073] With reference to the illustrative diagram in FIG. 5, assume thatthe N^(th) frame is one access (special) point for switching up.Normally the switching up only happens when the available channelbandwidth is high enough to cover the HB-MPFGS base layer bit rate.Here, S_(N) is the reconstructed image from the LB-MPFGS enhancementlayer at the corresponding bit rate. R_(N) is the reconstructed image atthe HB-MPFGS base layer. For the traditional method, the differencebetween them is losslessly compressed into the switching bitstream.

[0074] In the SF method, both S_(N) and R_(N) are first transformed intoDCT domain, and then are quantized with the same quantization parameter.The difference between two sets of quantized DCT coefficients iscompressed with the bit plane coding technique to form the switchingbitstream. In this case, the recovered image by the dequantized S_(N)plus the difference encoded in the switching bitstream is thedequantized {tilde over (R)}_(N). To avoid drifting errors at theHB-MPFGS base layer, the reference for the next frame coding is {tildeover (R)}_(N) instead of R_(N) in the switching frame.

[0075] Since the quantization is introduced in the SF method, theoverhead bits in the switching bitstream can be greatly reduced byadjusting the quantization parameter. One concern is the effect of theSF method on the coding efficiency of two MPFGS encoders. Firstly, S_(N)is used for the purposes of switching up. It is not used in the LB-MPFGScoding, thus the quantization of S_(N) will have no effect on the LBMPFGS coding efficiency. Using {tilde over (R)}_(N) as reference willaffect the coding efficiency of HB-MPFGS to some extent. However, it isexpected that the switching point/period can be inserted with about halfa second or about a one second interval in most applications. Otherintervals may also be used depending upon the design of the system.Here, it is assumed that this kind of coding efficiency losses isafforded in the HB-MPFGS.

[0076] Some Experimental Results:

[0077] Five different schemes were tested and compared in terms of bothcoding efficiency and channel bandwidth adaptation. The differentschemes included: (1) a seamless switch with lossless SB, (2) a seamlessswitch with SF, (3) MPEG-4 FGS, (4) MPFGS, and (5) a switch betweennon-scalable bitstreams.

[0078] For the experiment, a QCIF sequence News and Foreman were usedwith a 10 Hz encoding frame rate. Only the first frame was encoded as Iframe, and the subsequent frames were encoded as P frames. A TM5 ratecontrol method was used in the base layer encoding. For the experiment,the range of motion vectors was limited to ±15.5 pixel with half pixelprecision.

[0079] In the seamless scheme with or without the SF method, the bitrate of the LB-MPFGS base layer was 32 kbps. The high quality referencewas reconstructed at 64 kbps (e.g., the base layer plus a 32 kbpsenhancement layer), and channel bandwidth covered by LB-MPFGS was from32 kbps to 128 kbps. The bit rate of the HB-MPFGS base layer was 80 kbpsincluding the overhead bits for coding quantization parameters of theLB-MPFGS base layer. The high quality reference was reconstructed at 112kbps. The channel bandwidth range covered by the HB-MPFGS can be from 80kbps up to lossless rate. However, in this experiment, the upper boundof the HB-MPFGS bit rate was limited to 160 kbps.

[0080] Switching between non-scalable bitstreams is extensively used inmany commercial streaming video systems. Two non-scalable bitstreamswere used in this experiment with the same conditions as in the LB-MPFGSand HB-MPFGS base layers. However, I frames are inserted every tenframes for easy switching between bitstreams since channel bandwidthchanges with minimum 1 second interval in this experiment. In the singleMPEG-4 FGS and the single MPFGS schemes, the base layer bit rate wassame as that in LB-MPFGS. The high quality reference in the single MPFGSbitstream was reconstructed at the bit plane with bit rate over 40 kbps.Thus, most coding efficiency gain was biased toward high bit rates.

[0081] The effect of the SF technique on the coding efficiency of theHB-MPFGS base layer was first evaluated using graphs 600 and 610depicted in FIGS. 6A and 6B respectively, which shows a plot of thePSNR(dB) for a plurality of frames. Graph 600 is for a News Y QCIFbitstream and graph 610 is for a Foreman Y QCIF bitstream. Included ineach plot are two lines. Lines 602 and 612 are dashed lines showing theresults without the SF method, and lines 604 and 614 are solid linesshowing the results with the SF method. . The switching points/periodswere inserted every ten frames in the HB MPFGS base layer. Thequantization parameter of SF was set to 3 for all sequences.

[0082] The results in FIGS. 6A-B show that the average PSNR loss isbelow 0.1 dB compared with the lossless coding method. However, theaverage overhead bits in each switching frame are greatly reduced aslisted in the table below. Here, only about 20% overhead bits wereneeded for the SF technique. Average bits for each frame in the extrabitstream News Y QCIF Forman Y QCIF SF Lossless SF Lossless 30,351 bits133,090 bits 31,840 bits 156,212 bits

[0083] The plots of average PSNR(dB) versus bit rate (kbps) are depictedin graphs 700 and 720 in FIGS. 7A and & 7B, respectively. Graph 700 isfor a News Y QCIF bitstream and graph 720 is for a Foreman Y QCIFbitstream. Lines 702 and 722 illustrate the switch. Lines 704 and 724illustrate the results of MPFGS. Lines 706 and 726 illustrate theresults of the exemplary scheme provided above without SF. Lines 708 and728 illustrate the results of FGS. Lines 710 and 730 illustrate theresults of the exemplary scheme provided above with SF.

[0084] Switching between non-scalable bitstreams only provides twodifferent quality levels, while other schemes can flexibly and preciselyadapt to channel bandwidth and provide smooth visual quality. Comparedwith the single FGS and MPFGS bitstream schemes, the coding efficiencyof the exemplary scheme can be about 2.0 dB higher than MPFGS and about3.0 dB higher than FGS at higher bit rates. The SF method almostprovides the same performance as the lossless coding while greatlyreducing the overhead bits for switching up.

[0085] A dynamic channel was used to verify the performances of fourdifferent coding schemes in terms of bandwidth adaptation. The overheadbits for switching up were excluded in the simulation. In thisexperiment, the bit rate was periodically switched from 72 kbps to 152kbps. Each cycle starts at 72 kbps for 1 second and then switches to 152kbps for 3 seconds.

[0086] The results are plotted, PSNR(dB) versus frame number, in graphs800 and 820 in FIGS. 8A and 8B, respectively. Graph 800 is for a News YQCIF bitstream and graph 820 is for a Foreman Y QCIF bitstream. Lines802 and 822 illustrate the switch. Lines 804 and 824 illustrate theresults of MPFGS. Lines 808 and 828 illustrate the results of FGS. Lines810 and 830 illustrate the results of the exemplary scheme providedabove with SF.

[0087] The exemplary scheme switches up 3 times and switches down 2times in order to adapt channel bandwidth fluctuations. Clearly, theexemplary scheme with the SF technique achieves the best performancesamong these four schemes at both lower bit rates and higher bit rates.

[0088] Improved SP Coding Techniques:

[0089] In order to meet the requirements on file storage support, thejoint video Team (JVT) (a standards body) has adopted an SP codingtechnique in the current reference software. Some desiredfunctionalities are provided by the technique, such as bitstreamswitching, random access and bandwidth adaptation.

[0090]FIG. 9 is a block diagram illustrating a conventional SP codingscheme 900 used in the JVT software. The additional DCT transform andquantization operate on the prediction image. Since the current SPscheme at the encoder and decoder is quite different from that fornormal P frame, this makes the hardware implementation of JVT codec verycomplicated.

[0091]FIG. 10 is a block diagram illustrating an exemplary SP codingscheme 1000, in accordance with certain aspects of the presentinvention. SP coding scheme 1000 provides the same functionalitiesexisting in scheme 900. One difference between the two schemes, however,is the addition of improvement 1002 in which an additional DCT transformand quantization operate on the reconstructed image instead of theprediction image.

[0092] In order to switch between bitstreams in the SP frame, theoriginal SP coding scheme compresses the difference between twoprediction images into an additional bitstream, whereas the exemplary SPcoding scheme compresses the difference between two reconstructedimages.

[0093] Some further advantages of scheme 1000 are discussed in thefollowing content.

[0094] Simplification for Hardware Implementation:

[0095] As shown in FIG. 10, the additional modules for the SP coding areshown in improvement 1002. The other part is the same as that in the Pframe coding. When the current frame is a SP frame, the reconstructedimage is input to the SP modules. After quantization, the generatedimage is input to the Loop Filter modules and saved in a frame bufferfor the next frame coding. The quantization parameter in the module Qsis a unique value at frame level, which is encoded with the same methodin scheme 900, for example. It can be different from the parameter usedin the module Q. Therefore, for hardware implementation the exemplaryscheme 1000 can be made simpler by using the same codec for both P frameand SP frame.

[0096] Flexibility for Application Deployment:

[0097] Since different parameters can be used in the module Q and Qs,the exemplary scheme 1000 provides more flexibility for applications.Compared with the P frame, the coding efficiency loss in the SP frame iscaused by the quantization Qs. By adjusting the parameter in the moduleQs, a better trade-off between the number of switching bits and thecoding efficiency can be achieved.

[0098] An example is shown by graph 1100 in FIG. 11. Here, PSNR(dB)verses kbits/sec is shown, using five lines, with a Foreman Y QCIFbitstream as the source. Each line is based on six points correspondingto the quantization parameters 13, 16, 19, 22, 25 and 28 in the moduleQ. Except for the INTRA line 1102 and P line 1110, the other linesrepresent different quantization parameters in the module Qs. Here, line1104 is for SF-16, line 1106 is for SF-9, and line 1108 is for SF-3. Asshown, if the parameter is SF-3 (line 1108), the performance of SP isvery close to that of P (line 1110). As illustrated by these results,the exemplary scheme 1000 provides more flexibility.

[0099] In the conventional scheme 900, since the quantization operateson the prediction image, any lost information would directly affect thecoding efficiency of the current frame. However, in the exemplary scheme1000, the quantization operates on the reconstructed image. However, thedisplay image can still be reconstructed the same as the P frame. Thedegraded reference will affect the next frame. The coding efficiencyloss may be less than that in the current frame due to motioncompensation.

[0100] Conclusion

[0101] A seamless switching scheme for scalable video bitstreams hasbeen described that takes advantage of both the high coding efficiencyof non-scalable bitstreams and/or the flexibility of scalablebitstreams. Small bandwidth fluctuations are accommodated by thescalability of the bitstreams, while large bandwidth fluctuations aretolerated by switching between scalable bitstreams. The exemplary schemecan cover an extended bit rate range with significantly improved codingefficiency. Furthermore, no additional bits are needed in the freelyswitching down with the exemplary technique, and the exemplary SFtechnique also greatly reduces the overhead bits for switching up. Theexperimental results show that the exemplary scheme outperforms themethod of switching non-salable bitstreams both in coding efficiency andbandwidth adaptation. The coding efficiency of the exemplary scheme isalso significantly higher than that of single MPFGS or MPEG-4 FGSbitstream over a wide range of bit rates.

[0102] Although the description above uses language that is specific tostructural features and/or methodological acts, it is to be understoodthat the invention defined in the appended claims is not limited to thespecific features or acts described. Rather, the specific features andacts are disclosed as exemplary forms of implementing the invention.

What is claimed is:
 1. A method for switching between a first bitstreamand a second bitstream, wherein said second bitstream provides a higherdata rate than said first bitstream, said method comprising: identifyinga switching up period associated with said first and second bitstreams;establishing a third bitstream during said switching up period, saidthird bitstream being configured to carry additional data associatedwith a switch up from said first bitstream to said second bitstream; andswitching up from said first bitstream to said second bitstream duringsaid switching up period.
 2. The method as recited in claim 1, saidfirst bitstream comprising first encoded data for a plurality of frameswithin a video sequence, said first encoded data being at leastpartially transmitted in a base layer portion of said first bitstream.3. The method as recited in claim 2, wherein a portion of said firstencoded data is further transmitted in an enhanced layer portion of saidfirst bitstream.
 4. The method as recited in claim 3, wherein saidportion of said first encoded data transmitted in said enhanced layerportion of said first bitstream includes residue data associated withsaid first encoded data transmitted in said base layer portion.
 5. Themethod as recited in claim 4, wherein said residue data is based ondiscrete cosign transform (DCT) coefficients and dequantized (DCT)coefficients associated with said first encoded data transmitted in saidbase layer portion.
 6. The method as recited in claim 4, wherein saidresidue data is bit-plane coded.
 7. The method as recited in claim 2,said second bitstream comprising second encoded data for said pluralityof frames within said video sequence, said second encoded data being atleast partially transmitted in a base layer portion of said secondbitstream.
 8. The method as recited in claim 7, wherein a portion ofsaid second encoded data is further transmitted in an enhanced layerportion of said second bitstream.
 9. The method as recited in claim 8,wherein said portion of said second encoded data transmitted in saidenhanced layer portion of said second bitstream includes residue dataassociated with said second encoded data transmitted in said base layerportion.
 10. The method as recited in claim 9, wherein said residue datais based on discrete cosign transform (DCT) coefficients and dequantized(DCT) coefficients associated with said second encoded data transmittedin said base layer portion.
 11. The method as recited in claim 9,wherein said residue data is bit-plane coded.
 12. The method as recitedin claim 7, wherein said first encoded data transmitted in said baselayer portion of said first bitstream said second encoded datatransmitted in said base layer portion of said second bitstream are notgenerated independent of one another.
 13. The method as recited in claim12, wherein said first encoded data transmitted in said base layerportion of said first bitstream are based on reconstructed frame datausing at least said second encoded data transmitted in said base layerportion of said second bitstream.
 14. The method as recited in claim 13,wherein said first encoded data transmitted in said base layer portionof said first bitstream represents a difference between saidreconstructed frame data using at least said second encoded datatransmitted in said base layer portion of said second bitstream andtemporal prediction data.
 15. The method as recited in claim 12, whereinquantization parameters within said first encoded data transmitted insaid base layer portion of said first bitstream are further providedwithin said second encoded data transmitted in said base layer portionof said second bitstream.
 16. The method as recited in claim 15, furthercomprising switching down from said second bitstream to said firstbitstream using said quantization parameters.
 17. The method as recitedin claim 15, wherein said quantization parameters are adjusted at alevel selected from a group comprising a frame level, and a macroblocklevel.
 17. The method as recited in claim 12, wherein at least a portionof motion vector data associated with said second encoded data is alsooperatively associated with said first encoded data.
 18. The method asrecited in claim 7, wherein said first encoded data is encoded using acoding technique selected from a group comprising a Fine GranularityScalable (FGS) coding technique, a Progressive Fine Granularity Scalable(PFGS) coding technique, a macroblock-based PFGS (MPFGS) codingtechnique, an MPEG coding technique, an SF-Frame coding technique, and alossless coding technique.
 19. The method as recited in claim 7, furthercomprising: decoding said first encoded data prior to an end of saidswitching moment; and decoding said second encoded data following saidend of said switching moment.
 20. The method as recited in claim 19,wherein decoding said first encoded data includes using lower qualityreference data for prediction and reconstruction of said frames based onsaid first encoded data transmitted in said base layer portion of saidfirst bitstream.
 21. The method as recited in claim 20, wherein decodingsaid first encoded data further includes selectively using a higherquality reference data for prediction and reconstruction of said framesbased on said first encoded data transmitted in said enhancement layerportion of said first bitstream.
 22. The method as recited in claim 19,wherein decoding said second encoded data includes using lower qualityreference data for prediction and reconstruction of said frames based onsaid second encoded data transmitted in said base layer portion of saidsecond bitstream.
 23. The method as recited in claim 22, whereindecoding said second encoded data further includes selectively using ahigher quality reference data for prediction and reconstruction of saidframes based on said second encoded data transmitted in said enhancementlayer portion of said second bitstream.
 24. A computer-readable mediumhaving computer-executable instructions for performing acts comprising:receiving at least one bitstream; selectively switching between a firstbitstream and a second bitstream, wherein said second bitstream providesa higher data rate than said first bitstream by: identifying a switchingup period associated with said first and second bitstreams, establishinga third bitstream during said switching up period, said third bitstreambeing configured to carry additional data associated with a switch upfrom said first bitstream to said second bitstream; and switching upfrom said first bitstream to said second bitstream during said switchingup period.
 25. The computer-readable medium as recited in claim 24, saidfirst bitstream comprising first encoded data for a plurality of frameswithin a video sequence, said first encoded data being at leastpartially transmitted in a base layer portion of said first bitstream.26. The computer-readable medium as recited in claim 25, wherein aportion of said first encoded data is further transmitted in an enhancedlayer portion of said first bitstream.
 27. The computer-readable mediumas recited in claim 26, wherein said portion of said first encoded datatransmitted in said enhanced layer portion of said first bitstreamincludes residue data associated with said first encoded datatransmitted in said base layer portion.
 28. The computer-readable mediumas recited in claim 27, wherein said residue data is based on discretecosign transform (DCT) coefficients and dequantized (DCT) coefficientsassociated with said first encoded data transmitted in said base layerportion.
 29. The computer-readable medium -readable medium as recited inclaim 27, wherein said residue data is bit-plane coded.
 30. Thecomputer-readable medium as recited in claim 25, said second bitstreamcomprising second encoded data for said plurality of frames within saidvideo sequence, said second encoded data being at least partiallytransmitted in a base layer portion of said second bitstream.
 31. Thecomputer-readable medium as recited in claim 30, wherein a portion ofsaid second encoded data is further transmitted in an enhanced layerportion of said second bitstream.
 32. The computer-readable medium asrecited in claim 31, wherein said portion of said second encoded datatransmitted in said enhanced layer portion of said second bitstreamincludes residue data associated with said second encoded datatransmitted in said base layer portion.
 33. The computer-readable mediumas recited in claim 32, wherein said residue data is based on discretecosign transform (DCT) coefficients and dequantized (DCT) coefficientsassociated with said second encoded data transmitted in said base layerportion.
 34. The computer-readable medium as recited in claim 32,wherein said residue data is bit-plane coded.
 35. The computer-readablemedium as recited in claim 30, wherein said first encoded datatransmitted in said base layer portion of said first bitstream saidsecond encoded data transmitted in said base layer portion of saidsecond bitstream are not generated independent of one another.
 36. Thecomputer-readable medium as recited in claim 35, wherein said firstencoded data transmitted in said base layer portion of said firstbitstream are based on reconstructed frame data using at least saidsecond encoded data transmitted in said base layer portion of saidsecond bitstream.
 37. The computer-readable medium as recited in claim36, wherein said first encoded data transmitted in said base layerportion of said first bitstream represents a difference between saidreconstructed frame data using at least said second encoded datatransmitted in said base layer portion of said second bitstream andtemporal prediction data.
 38. The computer-readable medium as recited inclaim 35, wherein quantization parameters within said first encoded datatransmitted in said base layer portion of said first bitstream arefurther provided within said second encoded data transmitted in saidbase layer portion of said second bitstream.
 39. The computer-readablemedium as recited in claim 38, having computer-executable instructionsfor performing further acts comprising: switching down from said secondbitstream to said first bitstream using said quantization parameters.40. The computer-readable medium as recited in claim 38, wherein saidquantization parameters are adjusted at a level selected from a groupcomprising a frame level, and a macroblock level.
 41. Thecomputer-readable medium as recited in claim 35, wherein at least aportion of motion vector data associated with said second encoded datais also operatively associated with said first encoded data.
 42. Thecomputer-readable medium as recited in claim 30, wherein said firstencoded data is encoded using a coding technique selected from a groupcomprising a Fine Granularity Scalable (FGS) coding technique, aProgressive Fine Granularity Scalable (PFGS) coding technique, amacroblock-based PFGS (MPFGS) coding technique, an MPEG codingtechnique, an SF-Frame coding technique, and a lossless codingtechnique.
 43. The computer-readable medium as recited in claim 30,having computer-executable instructions for performing further actscomprising: decoding said first encoded data prior to an end of saidswitching moment; and decoding said second encoded data following saidend of said switching moment.
 44. The computer-readable medium asrecited in claim 43, wherein decoding said first encoded data includesusing lower quality reference data for prediction and reconstruction ofsaid frames based on said first encoded data transmitted in said baselayer portion of said first bitstream.
 45. The computer-readable mediumas recited in claim 44, wherein decoding said first encoded data furtherincludes selectively using a higher quality reference data forprediction and reconstruction of said frames based on said first encodeddata transmitted in said enhancement layer portion of said firstbitstream.
 46. The computer-readable medium as recited in claim 43,wherein decoding said second encoded data includes using lower qualityreference data for prediction and reconstruction of said frames based onsaid second encoded data transmitted in said base layer portion of saidsecond bitstream.
 47. The computer-readable medium as recited in claim46, wherein decoding said second encoded data further includesselectively using a higher quality reference data for prediction andreconstruction of said frames based on said second encoded datatransmitted in said enhancement layer portion of said second bitstream.48. An apparatus suitable for switching between a first bitstream and asecond bitstream, wherein said second bitstream provides a higher datarate than said first bitstream, said apparatus comprising: an interfaceconfigurable to selectively receive said first and second bitstreams;and logic operatively coupled to said interface and configured todetermine a switching up period associated with said first and secondbitstreams, cause said interface to support a third bitstream duringsaid switching up period, wherein said third bitstream is configured tocarry additional data associated with a switch up from said firstbitstream to said second bitstream, and cause said interface to switchup from receiving said first bitstream to receiving said secondbitstream as a result of said switch up.
 49. The apparatus as recited inclaim 48, said first bitstream comprising first encoded data for aplurality of frames within a video sequence, said first encoded databeing at least partially received in a base layer portion of said firstbitstream.
 50. The apparatus as recited in claim 49, wherein a portionof said first encoded data is further received in an enhanced layerportion of said first bitstream.
 51. The apparatus as recited in claim50, wherein said portion of said first encoded data received in saidenhanced layer portion of said first bitstream includes residue dataassociated with said first encoded data received in said base layerportion.
 52. The apparatus as recited in claim 51, wherein said residuedata is based on discrete cosign transform (DCT) coefficients anddequantized (DCT) coefficients associated with said first encoded datareceived in said base layer portion.
 53. The apparatus as recited inclaim 51, wherein said residue data is bit-plane coded.
 54. Theapparatus as recited in claim 49, said second bitstream comprisingsecond encoded data for said plurality of frames within said videosequence, said second encoded data being at least partially received ina base layer portion of said second bitstream.
 55. The apparatus asrecited in claim 54, wherein a portion of said second encoded data isfurther received in an enhanced layer portion of said second bitstream.56. The apparatus as recited in claim 55, wherein said portion of saidsecond encoded data received in s aid enhanced layer portion of saidsecond bitstream includes residue data associated with said secondencoded data received in said base layer portion.
 57. The apparatus asrecited in claim 56, wherein said residue data is based on discretecosign transform (DCT) coefficients and dequantized (DCT) coefficientsassociated with said second encoded data received in said base layerportion.
 58. The apparatus as recited in claim 56, wherein said residuedata is bit-plane coded.
 59. The apparatus as recited in claim 54,wherein said first encoded data received in said base layer portion ofsaid first bitstream said second encoded data received in said baselayer portion of said second bitstream are not generated independent ofone another.
 60. The apparatus as recited in claim 59, wherein saidfirst encoded data received in said base layer portion of said firstbitstream are based on reconstructed frame data using at least saidsecond encoded data received in said base layer portion of said secondbitstream.
 61. The apparatus as recited in claim 60, wherein said firstencoded data received in said base layer portion of said first bitstreamrepresents a difference between said reconstructed frame data using atleast said second encoded data received in said base layer portion ofsaid second bitstream and temporal prediction data.
 62. The apparatus asrecited in claim 59, wherein quantization parameters within said firstencoded data received in said base layer portion of said first bitstreamare further provided within said second encoded data received in saidbase layer portion of said second bitstream.
 63. The apparatus asrecited in claim 62, wherein said logic, using said quantizationparameters, is further configured to cause said interface to switch downfrom said second bitstream to said first bitstream.
 64. The apparatus asrecited in claim 62, wherein said quantization parameters are adjustedat a level selected from a group comprising a frame level, and amacroblock level.
 65. The apparatus as recited in claim 59, wherein atleast a portion of motion vector data associated with said secondencoded data is also operatively associated with said first encodeddata.
 66. The apparatus as recited in claim 54, wherein said firstencoded data is encoded using a coding technique selected from a groupcomprising a Fine Granularity Scalable (FGS) coding technique, aProgressive Fine Granularity Scalable (PFGS) coding technique, amacroblock-based PFGS (MPFGS) coding technique, an MPEG codingtechnique, an SF-Frame coding technique, and a lossless codingtechnique.
 67. The apparatus as recited in claim 54, wherein said logicis further configured to decode said first encoded data prior to an endof said switching moment, and said second encoded data following saidend of said switching moment.
 68. The apparatus as recited in claim 67,wherein said logic decodes said first encoded data includes using lowerquality reference data for prediction and reconstruction of said framesbased on said first encoded data received in said base layer portion ofsaid first bitstream.
 69. The apparatus as recited in claim 68, whereinsaid logic decodes said first encoded data further includes selectivelyusing a higher quality reference data for prediction and reconstructionof said frames based on said first encoded data received in saidenhancement layer portion of said first bitstream.
 70. The apparatus asrecited in claim 67, wherein said logic decodes said second encoded dataincludes using lower quality reference data for prediction andreconstruction of said frames based on said second encoded data receivedin said base layer portion of said second bitstream.
 71. The apparatusas recited in claim 70, wherein said logic decodes said second encodeddata further includes selectively using a higher quality reference datafor prediction and reconstruction of said frames based on said secondencoded data received in said enhancement layer portion of said secondbitstream.
 72. An apparatus suitable for encoding a first bitstream anda second bitstream, wherein said second bitstream provides a higher datarate than said first bitstream, said apparatus comprising: an interfaceconfigurable to selectively output said first and second bitstreams; andlogic operatively coupled to said interface and configured toselectively encode a plurality of frames of a video sequence within saidfirst bitstream as first encoded data that is at least partiallytransmittable by said interface in a base layer portion of said firstbitstream and said plurality of frames of said video sequence withinsaid second bitstream as second encoded data that is at least partiallytransmittable by said interface in a base layer portion of said secondbitstream, and wherein said logic establishes a plurality of temporallyspaced switching up periods associated with said first and secondbitstreams, during which said logic causes said interface to output athird bitstream wherein said third bitstream is configured to carryadditional data associated with a switch up from said first bitstream tosaid second bitstream.
 73. The apparatus as recited in claim 72, whereina portion of said first encoded data is further output in an enhancedlayer portion of said first bitstream.
 74. The apparatus as recited inclaim 73, wherein said portion of said first encoded data output in saidenhanced layer portion of said first bitstream includes residue dataassociated with said first encoded data received in said base layerportion.
 75. The apparatus as recited in claim 74, wherein said residuedata is based on discrete cosign transform (DCT) coefficients anddequantized (DCT) coefficients associated with said first encoded dataoutput in said base layer portion.
 76. The apparatus as recited in claim74, wherein said residue data is bit-plane coded.
 77. The apparatus asrecited in claim 72, said second bitstream comprising second encodeddata for said plurality of frames within said video sequence, saidsecond encoded data being at least partially received in a base layerportion of said second bitstream.
 78. The apparatus as recited in claim77, wherein a portion of said second encoded data is further output inan enhanced layer portion of said second bitstream.
 79. The apparatus asrecited in claim 78, wherein said portion of said second encoded dataoutput in said enhanced layer portion of said second bitstream includesresidue data associated with said second encoded data received in saidbase layer portion.
 80. The apparatus as recited in claim 79, whereinsaid residue data is based on discrete cosign transform (DCT)coefficients and dequantized (DCT) coefficients associated with saidsecond encoded data received in said base layer portion.
 81. Theapparatus as recited in claim 79, wherein said residue data is bit-planecoded.
 82. The apparatus as recited in claim 77, wherein said firstencoded data output in said base layer portion of said first bitstreamsaid second encoded data output in said base layer portion of saidsecond bitstream are not generated independent of one another.
 83. Theapparatus as recited in claim 82, wherein said first encoded data outputin said base layer portion of said first bitstream are based onreconstructed frame data using at least said second encoded datareceived in said base layer portion of said second bitstream.
 84. Theapparatus as recited in claim 83, wherein said first encoded data outputin said base layer portion of said first bitstream represents adifference between said reconstructed frame data using at least saidsecond encoded data received in said base layer portion of said secondbitstream and temporal prediction data.
 85. The apparatus as recited inclaim 82, wherein quantization parameters within said first encoded dataoutput in said base layer portion of said first bitstream are furtheroutput within said second encoded data received in said base layerportion of said second bitstream.
 86. The apparatus as recited in claim85, wherein said logic, using said quantization parameters, is furtherconfigured to cause said interface to switch down from said secondbitstream to said first bitstream.
 87. The apparatus as recited in claim85, wherein said quantization parameters are adjusted at a levelselected from a group comprising a frame level, and a macroblock level.88. The apparatus as recited in claim 82, wherein at least a portion ofmotion vector data associated with said second encoded data is alsooperatively associated with said first encoded data.
 89. The apparatusas recited in claim 77, wherein said first encoded data is encoded usinga coding technique selected from a group comprising a Fine GranularityScalable (FGS) coding technique, a Progressive Fine Granularity Scalable(PFGS) coding technique, a macroblock-based PFGS (MPFGS) codingtechnique, an MPEG coding technique, an SF-Frame coding technique, and alossless coding technique.
 90. The apparatus as recited in claim 77,further comprising memory operatively coupled to said logic andconfigured to store data associated with said logic.
 91. The apparatusas recited in claim 77, wherein said first encoded data includesproviding lower quality reference data for prediction and reconstructionof said frames within said first encoded data output in said base layerportion of said first bitstream.
 92. The apparatus as recited in claim91, wherein said first encoded data further includes selectively usablehigher quality reference data for prediction and reconstruction of saidframes within said first encoded data output in said enhancement layerportion of said first bitstream.
 93. The apparatus as recited in claim90, wherein said second encoded data includes lower quality referencedata for prediction and reconstruction of said frames within said secondencoded data output in said base layer portion of said second bitstream.94. The apparatus as recited in claim 91, wherein said second encodeddata further includes selectively usable higher quality reference datafor prediction and reconstruction of said frames based on said secondencoded data output in said enhancement layer portion of said secondbitstream.